Fostering Transparency in Outcomes, Quality, Safety, and Costs.

Public reporting of health care system performance is promoted as ameans for enhancing the value of health careby improvingquality and lowering costs.1 Transparency can improve value by engaging physicians and health careorganizations inquality improvementbyappealing to their professionalismandstimulating competitionamongorganizations,orbyprovidingpatientswith information that enables them to select physicians and health centers that offer higher-quality services, lowercostservices,orboth.2Thereareseveralexamplesofsuccess, such as the reporting of cardiac surgical outcomes from the Society of Thoracic Surgeons registry, reportingofmeasuresofhealthcare–associated infectionsfrom theCenters forDiseaseControlandPrevention,andmeasuresofpatientexperience,3but littleevidenceofbroad and sustained improvements. Thebenefits of transparencymaybeoffset by risks if the reporting is not valid. Today, there is no common standard for how reliable and valid a measure should be before it is publicly reported or used in a pay-forperformanceprogram.Usingmeasures forwhich the reliability andvalidity arepoororunknownposes risks, including disengaging clinicians from improvementwork and guiding patients to riskier care. It also raises potential ethical concerns, such as imposing unjust financial and reputational harmonphysicians andhealth careorganizations. For example, despitewidespreaduse, a recent assessment found that none of the 21 Agency for HealthcareResearchandQuality’s (AHRQ)PatientSafety Indicators met a basic threshold for accuracy.4 Over the last 20 years, the United States has witnessed a shift from having little information about the performance of the health care system to an abundance of measures reported in different ways bymany entities such as accreditation organizations, the Centers for Medicare &Medicaid Services (CMS), commercial health plans; consumer platforms (eg, Consumer Reports and Yelp), and independent parties including nonprofit and for-profit entities (eg, HealthGrades and USNewsandWorldReport).Thevarietyofmeasuresand methods, lack of standards, and failure to audit the underlying data can lead to conflicting results, which potentiallycreateconfusionordismissaloftheinformation.5 The current process of measuring and reporting the health care system’s performance on health, quality, safety,andcosts iserrorproneand lacksstandards.Data used for performance measurement are often first developed for a different purpose such as billing ormeeting regulatory requirements, so it is not surprising that these data can be problematicwhen used for purposes for which they were not developed. Each step in measuring and reporting performance presents opportunities for error. Yet no entity is entrusted with ensuring the validity of the whole process. First,measuresmust be developed and specified, which includes deciding on the dimension of care to be measured and identifying the target population, outcome, or process of interest, and, if appropriate, a riskadjustment model. The National Quality Forum (NQF) uses a multistakeholder consensus-development process to vet performancemeasures and endorses those deemed important, scientifically acceptable, feasible, and usable; use of these measures is voluntary. Second,datamustbeidentifiedandcollectedtopopulatethemeasures.WiththeexceptionoftheNationalCommitteeforQualityAssurance,whichevaluateshealthplan quality,someclinical registries,andasmallnumberofstate healthdepartments, fewofthedatausedforperformance measurement are subjected to quality-assurance procedures specific to the intendeduse formeasurement. Third, the collected data are applied to the measurespecifications.Entities that implementmeasuresoften indicate that they are using a standard endorsed measure, butdeviations fromthemeasure, differing interpretationsof themeasure specifications, andadjustments for convenience are common. Such variation in measure implementationmeans validity and comparability of the results are unknown. Fourth,apublic report iscreated,which includescategorizingcliniciansandhealthcareorganizations intoperformancegroups,andresultsarecommunicatedtostakeholders. Entities creating public reports also define performance categories, with no requirement that performance differences be tested for statistical significance. This can result inmisclassifying someclinicians or health care centers as better than orworse than others. Fifth,methods for communicating results to stakeholders have had little cognitive testing, leaving appropriate interpretability of the reports largely unknown.6